Detection of prohibited objects concealed in an item, using image processing

ABSTRACT

Some embodiments are directed to a system that includes a processor and memory circuitry (PMC) that is configured to: obtain an image of an item acquired by an acquisition device; perform a first detection using a first software module implementing at least one first deep neural network to detect at least one given area of the image including at least part of a given element of the item enabling concealment of a prohibited object; perform a second detection including using a second software module implementing at least one second neural network to detect whether the given area includes a prohibited object; and perform an action upon detection of a presence of a prohibited object in the image, wherein the detection is based at least on an output of the second detection.

TECHNOLOGICAL FIELD

The invention is in the field of detection of prohibited objectsconcealed in an item, using image processing.

BACKGROUND

At a security checkpoint (for example in an airport), an acquisitiondevice is used in order to acquire an image of an item carried by aperson. The image can be verified by an operator and/or by acomputerized system in order to detect the presence of a prohibitedobject (e.g. a weapon). Malicious persons use various techniques inorder to prevent detection of the prohibited object in their items.

There is therefore a need to propose new systems and methods toautomatically detect prohibited objects concealed in an item.

GENERAL DESCRIPTION

In accordance with certain aspects of the presently disclosed subjectmatter, there is provided a system comprising a processor and memorycircuitry (PMC) configured to obtain an image of an item acquired by anacquisition device, perform a first detection using a first softwaremodule implementing at least one first deep neural network, to detect atleast one given area of the image comprising at least part of a givenelement of the item enabling concealment of a prohibited object, performa second detection comprising using a second software moduleimplementing at least one second neural network to detect whether thegiven area comprises a prohibited object, and perform an action upondetection of a presence of a prohibited object in the image, whereinsaid detection is based at least on an output of said second detection.

In addition to the above features, the system according to this aspectof the presently disclosed subject matter can optionally comprise one ormore of features (i) to (xii) below, in any technically possiblecombination or permutation:

i. performing the second detection comprises detecting in the given areaa prohibited object which is fully or at least partially concealed usingthe given element;

ii. performing the action includes triggering an alert;

iii. performing the second detection only on a fraction of the imageincluding the given area;

iv. performing the first detection comprises determining datainformative of a type of the given element present in the given area;

v. the second software module implements a plurality of second neuralnetworks, wherein, for at least a subset of the plurality of secondneural networks, each given second neural network of the subset istrained to detect a prohibited object in an image comprising a differenttype of element enabling concealment of the prohibited object;

vi. the second software module implements a plurality of second neuralnetworks including a second neural network trained to detect in an imagea prohibited object fully or at least partially concealed using anelement of a first type; another second neural network trained to detectin an image a prohibited object fully or at least partially concealedusing an element of a second type, wherein the second type is differentfrom the first type;

vii. each given second neural network is trained using a training setcomprising: images in which a prohibited object is concealed using anelement of an item, the element being of a type for which the secondneural network is trained, and images in which no prohibited object ispresent;

viii. each given second neural network is trained using a training setcomprising only images in which no prohibited object is present;

ix. the system is configured to determine data informative of a giventype of the given element present in the given area, use the data toselect a given second neural network among a plurality of differentsecond neural networks implemented in the second software module,perform the second detection in a fraction of the image including thegiven area using the given second neural network;

x. for at least a subset of the plurality of second neural networks,each second neural network is trained to detect in an image presence ofa prohibited object concealed using an element of a different type, andthe system is configured to select the given second neural network whichis trained to detect in an image presence of a prohibited objectconcealed using an element of said given type;

xi. the system is configured to use the given second neural network todetect in the given area a prohibited object which is fully or at leastpartially concealed using the given element; and

xii. the system is configured to perform a detection in a majority ofthe image to detect whether a prohibited object is present in the image,thereby obtaining a first output informative of a presence of theprohibited object in the image, perform the first detection and thesecond detection, thereby obtaining a second output informative of apresence of a prohibited object in the image, perform an action upondetection of a presence of a prohibited object in the image, whereinsaid detection is based at least on the first output and the secondoutput.

In accordance with certain aspects of the presently disclosed subjectmatter, there is provided a method comprising, by a processor and memorycircuitry (PMC), obtaining an image of an item acquired by anacquisition device, performing a first detection using a first softwaremodule implementing at least one first deep neural network, to detect atleast one given area of the image comprising at least part of a givenelement of the item enabling concealment of a prohibited object,performing a second detection comprising using a second software moduleimplementing at least one second neural network to detect whether thegiven area comprises a prohibited object, and performing an action upondetection of a presence of a prohibited object in the image, whereinsaid detection is based at least on an output of said second detection.

In addition to the above features, the method according to this aspectof the presently disclosed subject matter can optionally implement oneor more of features (i) to (xii) as described with reference to thesystem above.

In accordance with certain aspects of the presently disclosed subjectmatter, there is provided a non-transitory storage device readable by amachine, tangibly embodying a program of instructions executable by themachine to perform operations as described with reference to the methodabove.

According to some embodiments, the proposed solution improves accuracyof detection of prohibited objects in an image of an item. As aconsequence, safety of persons and/or passengers is increased.

According to some embodiments, the proposed solution enables to detectprohibited objects which are concealed using concealing parts of an itemwhich are typical in the scanned item (e.g. handle metallic tubes,combination lock, metallic reinforcement strips and/or sheets of a bagor a suitcase, etc.). Although conventional methods face difficulties indetecting prohibited objects which are concealed using concealing partsof an item (since the concealing parts tend to hide, at least partially,the prohibited objects in the image), the proposed solution improvesdetection of the prohibited objects under such harsh conditions.

According to some embodiments, the proposed solution can be used withvarious types of acquisition devices.

According to some embodiments, the proposed solution improves accuracyof detection of prohibited objects while being computationallyefficient.

According to some embodiments, the proposed solution detects prohibitedobjects in real time or quasi real time.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to better understand the subject matter that is disclosedherein and to exemplify how it may be carried out in practice,embodiments will now be described, by way of non-limiting example only,with reference to the accompanying drawings, in which:

FIG. 1A illustrates an architecture of a system according to someembodiments of the invention;

FIGS. 1B to 1D illustrate non-limitative examples of items, eachincluding a concealing element enabling concealment of a prohibitedobject;

FIG. 1E illustrates a non-limitative example of an item in which aprohibited object is concealed using a concealing element in a baggage;

FIG. 2A illustrates a flow chart of an embodiment of a method ofdetecting concealing elements of an item in an image;

FIG. 2B illustrates a non-limitative example of data which can beobtained using the method of FIG. 2A;

FIG. 2C illustrates another non-limitative example of data which can beobtained using the method of FIG. 2A;

FIG. 3 illustrates a flow chart of an embodiment of a method of traininga neural network to detect concealing elements of an item in an image;

FIG. 4A illustrates a flow chart of an embodiment of a method ofdetecting presence of a prohibited object in an area corresponding to aconcealing element of an item;

FIG. 4B illustrates a variant of the method of FIG. 4A;

FIG. 4C illustrates a flow chart of an embodiment of a method ofdetecting presence of a prohibited object in an image of an item;

FIG. 5A illustrates a flow chart of an embodiment of a method oftraining a neural network to detect a prohibited object in an image of aconcealing element of an item;

FIGS. 5B and 5C illustrate images that can be used in the trainingmethod of FIG. 5A; and

FIG. 6 illustrates a flow chart of another embodiment of a method oftraining a neural network to detect a prohibited object in an image of aconcealing element of an item.

DETAILED DESCRIPTION OF EMBODIMENTS

In the following detailed description, numerous specific details are setforth in order to provide a thorough understanding of the invention.However, it will be understood by those skilled in the art that thepresently disclosed subject matter may be practiced without thesespecific details. In other instances, well-known methods have not beendescribed in detail so as not to obscure the presently disclosed subjectmatter.

The term “processor and memory circuitry” (PMC) as disclosed hereinshould be broadly construed to include any kind of electronic devicewith data processing circuitry, which includes for example a computerprocessing device operatively connected to a computer memory (e.g.digital signal processor (DSP), a microcontroller, a field programmablegate array (FPGA), and an application specific integrated circuit(ASIC), a graphics processing unit (GPU), etc.) capable of executingvarious data processing operations.

It can encompass a single processor or multiple processors, which may belocated in the same geographical zone, or may, at least partially, belocated in different zones and may be able to communicate together.

Unless specifically stated otherwise, as apparent from the followingdiscussions, it is appreciated that throughout the specificationdiscussions utilizing terms such as “obtaining”, “detecting”, “raising”,“determining”, “training”, “selecting”, “performing” or the like, referto the action(s) and/or process(es) of a processor and memory circuitrythat manipulates and/or transforms data into other data, said datarepresented as physical, such as electronic, quantities and/or said datarepresenting the physical objects.

In the specification, reference will be made to “prohibited objects”.This includes (but is not limited to) e.g. weapons (guns, knives,grenades, etc.), explosives (e.g. explosives which include a metallicmaterial, and/or shrapnel), etc. Although a typical example of aprohibited object is a threat object, this is however not limitative.The prohibited objects can include any object considered as prohibitedat a control or security checkpoint (according to rules set e.g. by anadministrator of the security checkpoint and/or by law) and shouldtherefore be detected. For example, a prohibited object in a facilitycan include e.g. a USB drive, a cellular phone, etc.

Embodiments of the presently disclosed subject matter are not describedwith reference to any particular programming language. It will beappreciated that a variety of programming languages may be used toimplement the teachings of the presently disclosed subject matter asdescribed herein.

The invention contemplates a computer program being readable by acomputer for executing one or more methods of the invention. Theinvention further contemplates a machine-readable memory tangiblyembodying a program of instructions executable by the machine forexecuting one or more methods of the invention.

Attention is drawn to FIG. 1A. FIG. 1A illustrates a system 100 whichcan be used at a control or security checkpoint, such as in an airport.This is however not limitative, and the system 100 can be used invarious other applications.

FIG. 1A illustrates an acquisition device 101, which is operative toacquire an image of an item 105.

The acquisition device 101 includes for example an X-ray acquisitiondevice, a scanner, a computerized tomography (CT) scan, or other typesof acquisition devices (e.g. body scanner). This list is not limitative.

Item 105 includes for example, a container, a bag, a piece of cloth, ashoe, etc. These examples are not limitative.

The acquisition device 101 is operatively connected to acomputer-implemented system 103. System 103 can be part of theacquisition device 101, or external to the acquisition device 101, orpartially part of the acquisition device 101 and partially external toit. System 103 is operative to receive an image 120 of the item 105 (ordata informative of the image 120) acquired by the acquisition device101.

System 103 includes a processor and memory circuitry (PMC) 104. PMC 104is configured to provide processing necessary for operating system 103,as further detailed in the various embodiments described hereinafter,and comprises a processor (not shown separately) and a memory (not shownseparately). System 103 can be used to perform various methods describedhereinafter, such as (but not limited to) the methods described withreference to FIGS. 2A, 3, 4A, 4B, 4C, 5A and 6 .

In FIG. 1A, PMC 104 is operatively connected to a hardware-based inputinterface 102 and to a hardware-based output interface 106. Theinterface 102 (e.g. a keyboard) can be used e.g. by an operator tointeract with system 103.

The processor of PMC 104 can be configured to execute several functionalmodules in accordance with computer-readable instructions implemented ona memory comprised in the PMC 104. Such functional modules are referredto hereinafter as included in the PMC 104.

A functional module comprised in PMC 104 includes a first softwaremodule 112. According to some embodiments, the first software module 112includes a list of instructions (e.g. an executable code/executableprogram) stored in a memory.

The instructions can encode operation of a model, such as a machinelearning network, and/or a sequence of computer vision instructionsand/or image processing instructions, and/or a deep neural network, orother adapted models.

In particular, the instructions are such that, when executed by a PMC(such as PMC 104), they cause the PMC to provide, based on an image ofan item received as an input (from e.g. the acquisition device 101),data informative of specific areas of the image. This will be discussedfurther in detail hereinafter.

According to some embodiments, and as mentioned above, the firstsoftware module 112 can implement a machine learning network. Thisincludes for example a neural network, such as a deep neural network(DNN).A deep neural network (DNN) comprises layers organized inaccordance with a respective DNN architecture. Optionally, at least someof the layers can be organized in a plurality of DNN sub-networks. Eachlayer of the ML network can include multiple basic computationalelements (CE), typically referred to in the art as dimensions, neurons,or nodes.

Generally, computational elements of a given layer can be connected withCEs of a preceding layer and/or a subsequent layer. Each connectionbetween a CE of a preceding layer and a CE of a subsequent layer isassociated with a weighting value. A given CE can receive inputs fromCEs of a previous layer via the respective connections, each givenconnection being associated with a weighting value which can be appliedto the input of the given connection. The weighting values can determinethe relative strength of the connections and thus the relative influenceof the respective inputs on the output of the given CE. The given CE canbe configured to compute an activation value (e.g. the weighted sum ofthe inputs) and further derive an output by applying an activationfunction to the computed activation. The activation function can be, forexample, an identity function, a deterministic function (e.g., linear,sigmoid, threshold, or the like), a stochastic function, or othersuitable function. The output from the given CE can be transmitted toCEs of a subsequent layer via the respective connections. Likewise, asabove, each connection at the output of a CE can be associated with aweighting value which can be applied to the output of the CE prior tobeing received as an input of a CE of a subsequent layer. Further to theweighting values, there can be threshold values (including limitingfunctions) associated with the connections and CEs.

The weighting and/or threshold values of the DNN can be initiallyselected prior to training, and can be further iteratively adjusted ormodified during training to achieve an optimal set of weighting and/orthreshold values in a trained ML network. After each iteration, adifference (also called loss function) can be determined between theactual output produced by ML network and the label or class associatedwith the respective training set of data. The difference can be referredto as an error value. Training can be determined to be complete when acost or loss function indicative of the error value is less than apredetermined value, or when a limited change in performance betweeniterations is achieved. Optionally, at least some of the ML subnetworks(if any) can be trained separately, prior to training the entire MLnetwork.

A set of ML network input data used to adjust the weights/thresholds ofa deep neural network is referred to hereinafter as a training set.

According to some embodiments, the machine learning network of the firstsoftware module 112 is used to implement a segmentation algorithm (e.g.a semantic segmentation algorithm, such as, but not limited to, U-net,Mask-RCNN), and/or an object detection algorithm (such as, but notlimited to, R-CNN, Region-Based Convolutional Neural Networks, FastR-CNN, and YOLO (You Only Look Once), etc.).

Another functional module comprised in PMC 104 includes a secondsoftware module 113. According to some embodiments, the second softwaremodule 113 includes a list of instructions (e.g. an executablecode/executable program) stored in a memory.

The instructions can encode operation of a model, such as a machinelearning algorithm, and/or a sequence of computer vision instructionsand/or image processing instructions, and/or a deep neural network, orother adapted models.

In particular, the instructions are such that, when executed by a PMC(such as PMC 104), they cause the PMC to provide, based on an image ofan item received as an input (from e.g. the acquisition device 100) anddata provided by the first software module 112, data informative of apresence of prohibited object(s) in the image. This will be discussedfurther in detail hereinafter.

According to some embodiments, and as mentioned above, the secondsoftware module 113 can implement at least one neural network (forexample a deep neural network).

In some embodiments, the second software module 113 implements aplurality of distinct neural networks 113 ₁ to 113 _(N). As explainedhereinafter, each neural network is trained using a different trainingset.

According to some embodiments, at least one neural network of the neuralnetworks 113 ₁ to 113 _(N) is used to implement a segmentation algorithm(e.g. semantic segmentation algorithm, such as, but not limited to,U-net, Mask-RCNN), and/or an object detection algorithm (such as, butnot limited to, R-CNN, Region-Based Convolutional Neural Networks, FastR-CNN, and YOLO (You Only Look Once)), and/or an anomaly detectionalgorithm (such as, via an Auto-Encoder and/or a Generative AdversarialNetwork).

This is not limitative, and other adapted object detectionalgorithms/neural networks can be used.

According to some embodiments, the neural networks 113 ₁ to 113 _(N) canbe organized according to a different architecture.

Although FIG. 1A depicts the first software module 112 and the secondsoftware module 113 as two different functional modules, in someembodiments, the first software module 112 and the second softwaremodule 113 can be implemented together in a same software module.

Upon processing the image 120, system 103 can send, via output interface106, data informative of the output of the processing to a device 107enabling a visual and/or audio representation of the processing. Device107 includes e.g. a screen and/or a loudspeaker. In some embodiments,system 103 can trigger an alert and/or send instructions to anotherdevice to trigger an alert.

It is noted that at least part of system 103 illustrated in FIG. 1A canbe implemented in a distributed computing environment, in which theaforementioned functional modules shown in FIG. 1A can be distributedover several local and/or remote devices and can be linked through acommunication network.

Attention is now drawn to FIGS. 1B to 1E.

FIG. 1B to 1E depict non-limitative examples of images of items 105,which can be acquired by the acquisition device 101.

The item comprises one or more elements (concealing element) enablingconcealment of a prohibited object in an image of the item (theprohibited object can be fully concealed, or at least partiallyconcealed—in some embodiments, a majority of the prohibited object isconcealed using the concealing element). In some embodiments, the itemincludes a plurality of prohibited objects, which can be of differenttypes, and can be concealed in the item using different concealingelements of the item.

The concealing elements enable a user to hide (at least partially) theprohibited object in an image of the item. The concealing element actsas “mask” and renders detection of the prohibited object more difficult.The concealing element typically includes a metallic material, althoughthis is not mandatory (e.g. thick organic substances, books and/or othermaterials). The material of the concealing element(s) attenuates theelectromagnetic waves transmitted by the acquisition device 101 (e.g.X-rays), and therefore the prohibited object is less visible in theimage than if it was not located above/underneath/within the concealingelement. Note that in an X-ray image, there is no real differencebetween an object located “above” the concealing element, “within” theconcealing element or “underneath” the concealing element, since in allcases the X-rays are attenuated by the concealing element beforereaching the detector of the X-ray machine, thereby making theprohibited object less visible in the image.

Non-limitative examples of concealing elements include:

-   -   baggage metallic tubes 130 ₁ (see FIG. 1B);    -   bag icon 130 ₂ (see FIG. 1D);    -   combination lock 130 ₃ (see FIG. 1D);    -   reinforcement strips 130 ₄ (or reinforcement sheets) of shoes        (see FIG. 1C).

FIG. 1E illustrates an example in which a knife 131 is concealed using abaggage metallic tube 130 ₁. As a consequence, visibility of the knife131 in the image is reduced.

Attention is now drawn to FIG. 2 .

Assume that an image of an item is obtained (operation 200). Asmentioned above, the image can be acquired by an acquisition device 101.

The method includes performing (operation 210) a first detection in theimage, to detect at least one given area of the image comprising atleast part of (or all of) a given element of the item enablingconcealment of a prohibited object using said given element (e.g. byconcealing the prohibited object using the given element).

In some embodiments, operation 210 can include detecting in the image aplurality of given areas. Each given area of the area comprises at leastpart of a given element of the item, enabling concealment of aprohibited object using said given element.

According to some embodiments, the first detection can be performedusing the first software module 112, which, as mentioned above,implements at least one (trained) first deep neural network.

FIG. 2B depicts an example of the output of FIG. 2A. An image 250 of abaggage is fed to the first deep neural network which identifies twodifferent given areas 260 and 261 of the image 250. Area 260 correspondsto a first baggage metallic tube of the baggage and area 261 correspondsto a second baggage metallic tube of the baggage.

In some embodiments, there can be an overlap in the image between afirst concealing element of the item and a second concealing element ofthe item (different from the first one). For example, in FIG. 2C, thecombination lock 262 and the baggage metallic tube 263 overlap in theimage. In some embodiments, the first deep neural network is trained todifferentiate between two (or more) different concealing elements, evenif they overlap in the image. In the example of FIG. 2B, this means thatthe first deep neural network identifies a first area 270 correspondingto the baggage metallic tube 263 in the image and a second area 271corresponding to the combination lock 262 in the image. The second area271 is located within the first area 270.

As shown in FIG. 2A, the first deep neural network outputs, at operation220, data informative of the given element(s) identified in the image,which correspond to concealing element(s) of the item acquired in theimage.

Data informative of the given element(s) can include a location/positionof the given element(s) in the image. For example, position of aplurality of points of the contour of each given element(s) in the imagecan be provided. This is however not limitative.

Data informative of the given element(s) can include a type of the givenelement. The type can be defined according to predefined categories.Examples of predefined categories can include: baggage metallic tubes,bag icon, combination lock, reinforcement strips (or reinforcementsheets) of shoes, etc. Note that other categories can be defined.

According to some embodiments, data informative of the given element caninclude type of the given element (as explained above) and type of theitem (e.g. bag, or piece of cloth, or shoes, etc.) in which the givenelement is located. This data can be determined e.g. by the trainedfirst deep neural network.

Type of the item can be determined based on image processing of thewhole image and/or based on a recognition of the given element itself.Generally, an object detection algorithm/neural network analyzes imageof the given element and image of the background surrounding the element(e.g. a shoe, a baggage, etc.) to determine the type of the item.

FIG. 3 depicts a method of training the first deep neural network. Insome embodiments, the training can include a supervised training, suchas (but not limited to Backpropagation).

The method includes obtaining (300) a training set including a pluralityof images. Each image includes an item (various examples of items havebeen provided above, and the images can include different items). Eachimage is associated with a label, which indicates a position of one ormore concealing elements of the item present in the image and which areusable to conceal prohibited object(s). The position of each concealingelement can be defined e.g. using the contour of the concealing elementin the image. The label can be obtained based on an input of an operatorwho annotates the image.

In some embodiments, the label can include additional data, such as thetype of the concealing element (see above examples of various possibledefinitions of types of concealing elements).

In some embodiments, the label can include data informative of the typeof the item itself (e.g. baggage, piece of cloth, etc.).

The method includes training (operation 310) the first deep neuralnetwork using the training set.

As explained above, the first deep neural network is trained to detecteach concealing element in the image, and, in particular, position ofeach concealing element in the image.

In some embodiments, the first deep neural network can be trained tofurther detect the type of each concealing element.

A set of types of concealing elements can be defined (e.g. by anoperator — see above various examples of types of concealing elements)and the first deep neural network can be trained to select, for eachconcealing element, the type which corresponds to the concealingelement. In some embodiments, the first deep neural network provides,for each type of the set of types, a probability that the concealingelement is of this type. The type which has the highest probability canbe selected.

In some embodiments, the first deep neural network can be trained tofurther detect the type of the item present in the image.

Once the first deep neural network is trained, it can be used in themethod of FIG. 2A. In some embodiments, the first deep neural networkcan be retrained periodically, using e.g. the real images provided tothe first deep neural network during operation, and the feedback of anoperator (who can indicate whether the output of the first deep neuralnetwork is correct).

Attention is now drawn to FIG. 4A.

As explained with reference to FIG. 2A, a first detection is performedon the image of the item to detect the area(s) of the image includingthe concealing elements.

The method of FIG. 4A can use at least part of the output (see operation400) of the method of FIG. 2A. In particular, the method of FIG. 4Aincludes performing a second detection using the second software module113.

The second software module 113 includes at least one second neuralnetwork (e.g. DNN) which is trained to detect prohibited objects in anarea including a concealing element of an item.

In some embodiments, each given area can be processed independently bythe second software module. As explained hereinafter, in someembodiments, at least some of the given areas can be processed bydifferent trained neural networks of the second software module.

For each given area which has been identified as including a concealingelement by the method of FIG. 2A, the method of FIG. 4A includesdetecting (operation 410) whether this given area includes a prohibitedobject.

In particular, according to some embodiments, the second detection isperformed only in a fraction of the image, which includes the givenarea(s) identified by the method of FIG. 2A. In other words, the seconddetection focuses on the given areas(s) identified by the firstdetection, thereby facilitating detection of the prohibited object(s).Each second neural network (which is used in the second detection)therefore receives as an input a given area (as explained hereinafter,in some embodiments, each given area can be fed to a second neuralnetwork which is selected as being specifically trained for detecting aprohibited object concealed using a type of element present in thisgiven area).

According to some embodiments, operation 410 includes using the givenarea of the image to detect presence of a prohibited object which isfully or at least partially concealed using the given element.

According to some embodiments, since each given area correspondssubstantially to a concealing element (all or at least the vast majorityof the given area corresponds to a concealing element), detectingwhether each given area includes a prohibited object enables detectingwhether a prohibited object is fully (or at least partially, or in itsmajority) concealed using the given area.

Operation 410 can include e.g. determining a prospect (e.g. probability)that a prohibited object is present. Based at least on this prospect, adecision whether a prohibited object is present in the image can bemade. For example, if the prospect is above a threshold (or equal to thethreshold), then a decision that a prohibited object is present is made,and if this is not the case, a decision that a prohibited object is notpresent can be made.

If it is detected that at least one given area (out of the plurality ofgiven areas) includes a prohibited object, an action can be performed(operation 420). Typically, performing an action can include raising analert. The alert can include e.g. a textual alert and/or visual alertand/or audio alert. The alert can be displayed e.g. on a screen, and/ortransmitted to an electronic device (e.g. computer, smartphone) of oneor more operators.

In some embodiments, raising an alert can include sending a command toanother system which can trigger the alert.

In some embodiments, performing an action can include associating, in adatabase, identity data of the item's owner with a malicious label.

In some embodiments, performing an action can include triggering asystem to destroy or annihilate the item and/or the detected prohibitedobject.

In some embodiments, the action and/or alert can be different dependingon the type of prohibited object. For example, for a prohibited objectconsidered as highly dangerous (e.g. an assault rifle), a first type ofalert can be raised, and for a prohibited object considered as lessdangerous (e.g. scissors), a second type of alert can be raised,different from the first type.

In some embodiments, the action and/or alert can include instructing aconveyor (of the lane) to move the item to a separate area for manualinspection.

In some embodiments, the action and/or alert can include sending aninstruction (e.g. to the acquisition device) to immediately stop theconveyor, while the item is still inside the tunnel. This can be usede.g. when a bomb (or another similar dangerous object) has beendetected.

In some embodiments, the method can include outputting additional datainformative of the prohibited object(s). This data can include locationof the prohibited object in the image.

In some embodiments, the method can output a modified image in whichlocation of the prohibited object is emphasized (using e.g. device 107),thereby facilitating manual inspection of the item by an operator. Thisis shown in FIG. 1E, in which a bounding box 132 is added on the imageand indicates presence of a knife (concealed using the concealingelement 130 ₁) identified during the second detection.

In some embodiments, data output by the method can include e.g. the sizeand/or shape and/or type (e.g. knife, rifle, etc.) of the prohibitedobject. This data can be provided by the second neural network, whichcan be trained to provide such data, as explained hereinafter.

Attention is now drawn to FIG. 4B.

The method of FIG. 4B includes obtaining data informative of the givenarea(s) including the concealing element(s) (operation 400, alreadydescribed above with reference to FIG. 4A).

In this embodiment, the second software module implements a plurality ofsecond neural networks 113 ₁ to 113 _(N) (e.g. deep neural networks).

Each second neural network 113 ₁ to 113 _(N) is trained to detectpresence of a prohibited object in an image in which the prohibitedobject is (fully or at least partially, or in its majority) concealedusing a concealing element of an item.

According to some embodiments, each second neural network of theplurality of second neural networks is trained to detect a prohibitedobject in an image which includes a different type of concealingelement.

More particularly, for at least a subset of the plurality of secondneural networks, or for all of them, each second neural network of theplurality of second neural networks is trained to detect in an image aprohibited object concealed (fully or at least partially or in itsmajority) using a concealing element of a different type.

For example, second neural network 113 ₁ is trained to detect prohibitedobject(s) concealed using a first type of concealing element, secondneural network 113 ₂ is trained to detect prohibited object(s) concealedusing a second type of concealing element (different from the firsttype), etc.

It can occur than some of the second neural networks of the plurality ofneural networks are trained to detect prohibited object(s) concealedusing the same type of concealing element (for example two second neuralnetworks are organized according to a different architecture but aretrained to detect prohibited objects concealed using the same type ofconcealing element).

The different types of concealing elements can be defined according tovarious rules.

The type can be defined according to predefined categories. Examples ofpredefined categories can include: baggage metallic tubes, bag icon,combination lock, reinforcement strips (or reinforcement sheets) ofshoes, etc. In some embodiments, the categories can be split into anumber of sub-categories, in order to further improve accuracy. Forexample, baggage metallic tubes comprise a first subcategory of baggagemetallic tubes of small size baggage, a second subcategory of baggagemetallic tubes of intermediate size baggage, and a third subcategory ofbaggage metallic tubes of large size baggage. This is however notlimitative. Other categories/subcategories can be defined, depending onthe needs.

According to some embodiments, the type of concealing element can bedefined with reference to the type of the item. For example, a firsttype corresponds to a reinforcement strip of shoes and a second typecorresponds to a reinforcement strip of a bag.

According to some embodiments, the type of concealing element can bedefined with reference to the material of the concealing element. Forexample, if the X-ray image is a colored image, a table linking color inthe image to the atomic number of the material can be used toautomatically detect type of concealing element in the image. Note thatin case a neural network is used for object detection, the color of theimage is automatically taken into account to detect the type ofconcealing element.

As mentioned above with reference to FIG. 2A, data informative of thetype of the concealing element present in each area can be determined bythe first software module.

The method of FIG. 4B includes, for each given area of the imageidentified as including a concealing element, using data informative ofthe type of concealing element to select a given second neural networkamong the plurality of different second neural networks. Assume that agiven area includes a concealing element of a given type. Operation 425can include selecting the given second neural network which has beenspecifically trained to detect a prohibited object in an area of animage including a concealing element of this given type.

For example, assume that it has been determined that a given area of theimage includes a baggage metallic tube. Assume that the second softwaremodule implements at least one second neural network which has beentrained to detect prohibited objects concealed using this type ofconcealing element (baggage metallic tube). As a consequence, thissecond neural network is selected to perform the second detection, andto verify whether a prohibited object is present in this given area.

The method further includes (operation 430), for each given area,detecting whether a prohibited object is present in the given area usingthe second neural network selected for this given area. Since the givenarea corresponds substantially to the given concealing element,operation 430 enables detecting a prohibited object which is fully (orat least partially, or in its majority) concealed using the givenconcealing element.

If a plurality of different areas has been identified in the image, andeach area includes a different type of concealing element, then adifferent second neural network is selected and used to perform thesecond detection in each area.

Upon detection of presence of a prohibited object in the image, anaction can be performed (operation 440, which is similar to operation420 already described above).

Attention is now drawn to FIG. 4C.

The method of FIG. 4C includes detecting (operation 450) whether aprohibited object is present based on an analysis of the whole image (ora majority of the image), thereby obtaining a first output (e.g. ascore/prospect/probability) informative of the presence of a prohibitedobject. For example, a deep neural network can be trained to detectpresence of a prohibited object in an image of an item, using e.g.supervised learning (a training set of images including prohibitedobjects and images which do not include prohibited objects can be used).In some embodiments, for each type of item (e.g.

baggage, shoes, etc.), a different deep neural network can be trainedand used in the method of FIG. 4C.

In operation 450, the image is processed as a whole.

The method of FIG. 4C further includes detecting (operation 460)presence of given areas corresponding to concealing elements of theitem. Operation 460 can be performed using the method of FIG. 2A.

The method of FIG. 4C further includes determining whether a prohibitedobject is present in each given area (as mentioned above, this includesdetecting presence of a prohibited object fully or at least partiallyconcealed using a concealing element present in the given area).Operation 460 can be performed using e.g. the method of FIG. 4A or FIG.4B. Operation 460 enables to obtain a second output (e.g. ascore/prospect/probability) informative of the presence of a prohibitedobject.

The method of FIG. 4C further includes performing the action upondetection of the presence of a prohibited object in the image, whereinthe detection is based at least on the first output and the secondoutput.

In other words, the detection depicted in FIG. 4C is based both on aglobal approach (in which the whole image is processed), and an approachwhich focuses on the concealing elements.

Assume that the first output provides a first probability that aprohibited object is present and that the second output provides asecond probability that a prohibited object is present. An aggregatedprobability can be computed and compared to a threshold. In someembodiments, a weight can be assigned to the second probability which ishigher than a weight assigned to first probability, since the secondprobability is more accurate.

If the aggregated probability is above or equal to the threshold, thisindicates that a prohibited object is present. If not, this indicatesthat the prohibited object is not present.

In some embodiments, operation 450 provides, for one or more firstlocations in the image, a probability that a prohibited object ispresent.

Similarity, operation 470 provides, for one or more second locations inthe image, a probability that a prohibited object is present.

For each location of the image, a probability that a prohibited objectis present can be computed based on the output of operation 450 and/oron the output of operation 470. If for at least one location thecomputed probability is equal to or above a threshold, an action can beperformed (operation 480), as already explained with reference to FIG.4A.

Attention is now drawn to FIG. 5 .

As mentioned above, according to some embodiments, each given secondneural network (113 ₁ to 113 _(N)) is trained to detect a prohibitedobject partially or fully concealed using a concealing element of agiven type.

FIG. 5A depicts a non-limitative method of training a given secondneural network for detection with respect to a given type of concealingelement.

The method includes (operation 500) obtaining a training set. Thetraining set includes:

a plurality of first images, each first image including a concealingelement of the given type, and a prohibited object fully (or at leastpartially, or in its majority) concealed using the concealing element ofthe given type (see e.g. image 550 in FIG. 5B, in which a knife 550 ₁ isfully concealed using baggage metallic tube 550 ₂);

a plurality of second images, wherein each second image includes aconcealing element of the given type but does not include a prohibitedobject (see e.g. image 551 in FIG. 5C, in which no prohibited object ispresent under or above or within the baggage metallic tube 550 ₃).

At least some of the images of the training set can be real imagesand/or real images in which a prohibited object has been artificiallyadded and/or simulated images and/or synthetic images.

According to some embodiments, the first images include different typesof prohibited objects, in order to train the second neural network todetect various types of prohibited objects.

According to some embodiments, the training set includes imagesdepicting a given type of concealing element used in different types ofitems.

Each image of the training set is labelled (e.g. by an operator). Thelabel indicates e.g. whether the image includes a prohibited object. Insome embodiments, the label can include further data such as the type ofprohibited object, its location, the type of concealing element, etc.

According to some embodiments, each image of the training set focuses onthe concealing element, meaning that all, or at least the majority ofthe image, includes the concealing element.

The method of FIG. 5A further includes using the training set to train agiven second neural network of the plurality of second neural networks113 ₁ to 113 _(N). Training can use methods such as Backpropagation(this is however not limitative). The given second neural network istherefore trained specifically for detecting, in an image, prohibitedobjects concealed using a concealing element of the given type.

In some embodiments, each second neural network is trained to output aprobability that a prohibited object is present.

The method of FIG. 5A can be repeated for each second neural network ofthe plurality of second neural networks 113 ₁ to 113 _(N). A differenttraining set can be used for each second neural network. In particular,each training set can be dedicated to a different type of concealingelement. As a consequence, each second neural network is trained todetect a prohibited object concealed using a different type ofconcealing element.

Attention is drawn to FIG. 6 .

The method includes (operation 600) obtaining a training set. Thetraining set includes a plurality of images. Each image of the trainingset (or at least most of them) includes a concealing element of a giventype which does not include a prohibited object (see e.g. image 551 inFIG. 5C). The images can be selected such that they include differenttypes of concealing elements.

At least some of the images of the training set can be real imagesand/or simulated images and/or synthetic images.

According to some embodiments, the images include different types ofprohibited objects, in order to train the second neural network todetect various types of prohibited objects.

According to some embodiments, the training set includes a given type ofconcealing element, but for different types of items.

According to some embodiments, each image of the training set focuses onthe concealing element, meaning that all, or at least the majority ofthe image, includes the concealing element.

In this method, it is not required to label the images.

According to some embodiments, each image of the training set focuses onthe concealing element, meaning that all, or at least the majority ofthe image, includes the concealing element.

The method of FIG. 6 further includes training (operation 610) a givensecond neural network of the plurality of second neural networks 113 ₁to 113 _(N), using the training set obtained at operation 600.

Training can be performed using anomaly detection methods. This ishowever not limitative.

The method of FIG. 6 is particularly beneficial when the second neuralnetwork is an Auto-Encoder, or a Generative Adversarial Network (the GANcan be combined with a discriminator). This is however not limitative,and other deep neural networks can be used.

The given second neural network is therefore trained specifically fordetecting, in an image, prohibited objects concealed using a concealingelement of the given type. The given second neural network is trained todetect anomalies Indeed, since it has been trained on images which arefree of prohibited objects, if, during the prediction stage, itencounters an image with a prohibited object, it is able to provide ascore informative of the probability of a presence of a prohibitedobject (presence of the prohibited object is viewed as an anomaly by thesecond neural network). Since a large training set of data is used(which covers various scenarios including unprohibited objects), thisgenerally prevents from raising an alarm when an unprohibited object isconcealed using the concealing element.

The method of FIG. 6 can be repeated for each second neural network ofthe plurality of second neural networks 113 ₁ to 113 _(N). A differenttraining set can be used for each second neural network. In particular,each training set can be dedicated to a different type of concealingelement.

As can be understood from the methods described above, training eachgiven second neural network to specifically detect prohibited objectsconcealed using a given type of concealing element can be obtained usinga training set in which all, or at least a majority of the images of thetraining set, includes the concealing element of the given type. In themethod of FIG. 5A, both positive examples (with a prohibited object) andnegative examples (without a prohibited object) are used in the trainingset, and in the method of FIG. 6 most or all of the examples of thetraining set are negative examples.

It is to be noted that the various features described in the variousembodiments may be combined according to all possible technicalcombinations.

It is to be understood that the invention is not limited in itsapplication to the details set forth in the description contained hereinor illustrated in the drawings. The invention is capable of otherembodiments and of being practiced and carried out in various ways.Hence, it is to be understood that the phraseology and terminologyemployed herein are for the purpose of description and should not beregarded as limiting. As such, those skilled in the art will appreciatethat the conception upon which this disclosure is based may readily beutilized as a basis for designing other structures, methods, and systemsfor carrying out the several purposes of the presently disclosed subjectmatter.

Those skilled in the art will readily appreciate that variousmodifications and changes can be applied to the embodiments of theinvention as hereinbefore described without departing from its scope,defined in and by the appended claims.

1-27. (canceled)
 28. A system comprising a processor and memorycircuitry (PMC) configured to: obtain an image of an item acquired by anacquisition device, perform a first detection using a first softwaremodule implementing at least one first deep neural network, to detect atleast one given area of the image comprising at least part of a givenelement of the item enabling concealment of a prohibited object, performa second detection comprising using a second software moduleimplementing at least one second neural network to detect whether thegiven area comprises a prohibited object, and perform an action upondetection of a presence of a prohibited object in the image, whereinsaid detection is based at least on an output of said second detection.29. The system of claim 28, wherein performing the second detectioncomprises detecting in the given area a prohibited object which is fullyor at least partially concealed using the given element.
 30. The systemof claim 28, wherein performing the action includes triggering an alert.31. The system of claim 28, configured to perform the second detectiononly on a fraction of the image including the given area.
 32. The systemof claim 28, wherein performing the first detection comprisesdetermining data informative of a type of the given element present inthe given area.
 33. The system of claim 28, wherein the second softwaremodule implements a plurality of second neural networks, wherein, for atleast a subset of the plurality of second neural networks, each givensecond neural network of the subset is trained to detect a prohibitedobject in an image comprising a different type of element enablingconcealment of the prohibited object.
 34. The system of claim 28,wherein the second software module implements a plurality of secondneural networks including: a second neural network trained to detect inan image a prohibited object fully or at least partially concealed usingan element of a first type; another second neural network trained todetect in an image a prohibited object fully or at least partiallyconcealed using an element of a second type, wherein the second type isdifferent from the first type.
 35. The system of claim 33, wherein eachgiven second neural network has been trained using a training setcomprising: images in which a prohibited object is concealed using anelement of an item, the element being of a type for which the secondneural network is trained, and images in which no prohibited object ispresent.
 36. The system of claim 33, wherein each given second neuralnetwork has been trained using a training set comprising only images inwhich no prohibited object is present.
 37. The system of claim 28,configured to: determine data informative of a given type of the givenelement present in the given area, use the data to select a given secondneural network among a plurality of different second neural networksimplemented in the second software module, perform the second detectionin a fraction of the image including the given area using the givensecond neural network.
 38. The system of claim 37, wherein: for at leasta subset of the plurality of second neural networks, each second neuralnetwork is trained to detect in an image presence of a prohibited objectconcealed using an element of a different type, and the system isconfigured to select the given second neural network which is trained todetect in an image presence of a prohibited object concealed using anelement of said given type.
 39. The system of claim 37, configured touse the given second neural network to detect in the given area aprohibited object which is fully or at least partially concealed usingthe given element.
 40. The system of claim 28, configured to: perform adetection in a majority of the image to detect whether a prohibitedobject is present in the image, thereby obtaining a first outputinformative of a presence of the prohibited object in the image, performthe first detection and the second detection, thereby obtaining a secondoutput informative of a presence of a prohibited object in the image,perform an action upon detection of a presence of a prohibited object inthe image, wherein said detection is based at least on the first outputand the second output.
 41. A method comprising, by a processor andmemory circuitry (PMC): obtaining an image of an item acquired by anacquisition device, performing a first detection using a first softwaremodule implementing at least one first deep neural network, to detect atleast one given area of the image comprising at least part of a givenelement of the item enabling concealment of a prohibited object,performing a second detection comprising using a second software moduleimplementing at least one second neural network to detect whether thegiven area comprises a prohibited object, and performing an action upondetection of a presence of a prohibited object in the image, whereinsaid detection is based at least on an output of said second detection.42. The method of claim 41, comprising performing at least one of (i) or(ii) or (iii) or (iv): (i) performing the second detection comprisesdetecting in the given area a prohibited object which is fully or atleast partially concealed using the given element, or (ii) performingthe second detection only on a fraction of the image including the givenarea, or (iii) performing the first detection comprises determining datainformative of a type of the given element present in the given area, or(iv) using the given second neural network to detect in the given area aprohibited object which is fully or at least partially concealed usingthe given element.
 43. The method of claim 41, wherein at least one of(i) or (ii) is met: (i) the second software module implements aplurality of second neural networks, wherein for at least a subset ofthe plurality of second neural networks, each given second neuralnetwork of the subset is trained to detect a prohibited object in animage comprising a different type of element enabling concealment of theprohibited object, or (ii) the second software module implements aplurality of second neural networks including: a second neural networktrained to detect in an image a prohibited object fully or at leastpartially concealed using an element of a first type; another secondneural network trained to detect in an image a prohibited object fullyor at least partially concealed using an element of a second type,wherein the second type is different from the first type.
 44. The methodof claim 41, wherein at least one of (i) or (ii) is met: (i) each givensecond neural network has been trained using a training set comprising:images in which a prohibited object is concealed using an element of anitem, the element being of a type for which the given second neuralnetwork is trained, and images in which no prohibited object is present,or (ii) each given second neural network has been trained using atraining set comprising only images in which no prohibited object ispresent.
 45. The method of claim 41, comprising performing at least oneof (i) or (ii): (i) determining data informative of a given type of thegiven element present in the given area, using the data to select agiven second neural network among a plurality of different second neuralnetworks implemented in the second software module, and performing thesecond detection in a fraction of the image including the given areausing the given second neural network, or (ii) determining datainformative of a given type of the given element present in the givenarea, for at least a subset of a plurality of second neural networksimplemented in the second software module, each second neural network istrained to detect in an image presence of a prohibited object concealedusing an element of a different type, wherein the method comprisesselecting the given second neural network which is trained to detect inan image presence of a prohibited object concealed using an element ofsaid given type.
 46. The method of claim 41, comprising: performingdetection in a majority of the image to detect whether a prohibitedobject is present in the image, thereby obtaining a first outputinformative of a presence of the prohibited object in the image,performing the first detection and the second detection, therebyobtaining a second output informative of a presence of a prohibitedobject in the image, and performing an action upon detection of apresence of a prohibited object in the image, wherein said detection isbased at least on the first output and the second output.
 47. Anon-transitory storage device readable by a processor and memorycircuitry (PMC), tangibly embodying a program of instructions executableby the PMC to perform: obtaining an image of an item acquired by anacquisition device, performing a first detection using a first softwaremodule implementing at least one first deep neural network, to detect atleast one given area of the image comprising at least part of a givenelement of the item enabling concealment of a prohibited object,performing a second detection comprising using a second software moduleimplementing at least one second neural network to detect whether thegiven area comprises a prohibited object, and performing an action upondetection of a presence of a prohibited object in the image, whereinsaid detection is based at least on an output of said second detection.